Skip to content

Increase file cache chunk size from 16 KB to 1 MB#1388

Merged
veluca93 merged 1 commit into
cms-dev:mainfrom
veluca93:filecacher-large-files
Jun 8, 2025
Merged

Increase file cache chunk size from 16 KB to 1 MB#1388
veluca93 merged 1 commit into
cms-dev:mainfrom
veluca93:filecacher-large-files

Conversation

@veluca93

@veluca93 veluca93 commented Jun 7, 2025

Copy link
Copy Markdown
Contributor

To improve performance when connecting to worker instances over high latency network

Fixes #1368

@veluca93
veluca93 force-pushed the filecacher-large-files branch from 8228ddb to 69f4818 Compare June 7, 2025 22:27
@prandla

prandla commented Jun 8, 2025

Copy link
Copy Markdown
Member

I think it would be nice to benchmark this (with different amounts of simulated network latency between cms and the database), but i'm pretty sure using 1MB as the default won't really break anything. I do think that making it too big might make things slower in some cases, but maybe 1MB isn't noticeable yet.

@veluca93

veluca93 commented Jun 8, 2025

Copy link
Copy Markdown
Contributor Author

I'm also fairly sure this will be fine at 1mb - and on low latency networks I imagine it won't cause problems.

On high latency networks, I think this was tested enough at IOI :D

To improve performance when connecting to worker instances over high latency network
@veluca93
veluca93 force-pushed the filecacher-large-files branch from 69f4818 to 399c75e Compare June 8, 2025 17:01
@veluca93
veluca93 merged commit 83685ee into cms-dev:main Jun 8, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

[IOI fork parity] increase file cacher chunk size

3 participants